Conventional lossy compression schemes use quantization to reduce a bit-rate of data samples. Quantization is commonly used to compress (i) original pulse code modulated samples of a signal, (ii) transform coefficient samples of images (i.e., JPEG, MPEG and H.264) and (iii) differential pulse code modulated samples of a signal. Input data samples subjected to quantization and then inverse quantization often result in output data samples that are different from the input data samples. The differences are typically the only lossy part of lossy compression schemes.
The subjective loss of quality caused by the quantization does not depend only on an absolute difference between the decompressed samples and the original (pre-compressed) samples. For example, if the compression causes many adjacent samples to have identical values after decompression, artificial contours can appear in the decompressed signal.
Referring to FIG. 1, a Table I illustrating an example conventional quantization and dequantization of data samples is shown. Using a step size of 8, the quantization of an original sample X is calculated as Q=floor(X/8). A decoder using a dequantization offset of 4 will decode a quantized value of Q with the formula R=Q×8+4, where R is the dequantized data sample.
An error range between the original samples X and the dequantized samples R is [−3, 4]. Table I shows that for random original values X, the errors of −3, −2, −1, 0, 1, 2, 3 and 4 have a similar probability. Therefore, the average absolute error in the example is (|−3|+|−2|+|−1|+|0|+|1|+|2|+|3|+|3|+|4|)/8=2.
Referring to FIG. 2, a graph 20 of the data in Table I is shown. When the above example quantization scheme is applied to a slowly varying signal, the quantization may lead to abrupt jumps in the decompressed signal. In the example, X increases by 1 each step along a line 22. A first set of samples X are quantized and dequantized to a first value R=100, as indicated by reference 24. A second set of samples X are quantized and dequantized to a second value R=108, as indicated by reference 26, and so on. As can be seen from the graph 20, the values R have abrupt jumps that create a staircase like effect. The staircase like effect can have unfavorable consequences. For example, if the signal being compressed is image data, the jumps in the values of R will appear as visible contours in the decompressed picture, even though the original picture had no such contours.
It is desirable to have a lossy compression scheme that does not cause sharp contours in the decompressed signal.